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A VIDEO BUS FOR HIGH SPEED MULTI-RESOLUTION IMAGERS 



This application is a continuation-in-part application of Application Serial 
No. 09/039,835 filed on March IM^' ^ < ^ 

Field of the Invention : 

This invention relates generally to busses and, more particularly, to a video 
bus for high speed multi-resolution imagers. 



10 Background of the Invention : 

An active pixel is a semiconductor device capable of converting an optical 
image into an electronic signal. Active pixels can be arranged in a matrix and 
utilized to generate video signals for video cameras, still photography, or 
anywhere incident radiation needs to be quantified. When incident radiation 

1 5 interacts with a photosite, charge carriers are liberated and can be collected for 

sensing. The number of carriers collected in a photosite represents the amount of 
incident light impinging on the site in a given time-period. 

There are two basic devices with many variants, employed to collect and 
sense, charge carriers in a photosite. The two basic devices are photodiodes and 

20 photogates. Variants of photodiodes include, but are not limited to: Pinned, P-I- 
N, Metal- Semiconductor, Heteroj unction, and Avalanche. Photogate structures 
include: Charge Couple Devices (CCD), Charge Injection Devices (CID) and 
their variants that include virtual phase, buried channel and other variations that 
utilize selective dopants. The selective dopants are used to control charge 

25 collection and transfer underneath and between the photogate(s) and the sense 
node. 

The solid state imagers heretofore used have been dominated by CCD's 
because of their low noise as compared to Photodiodes and CIDs. The low noise 
advantage of CCD imagers is the result of collecting the photon generated charge 
30 at the pixel site and then coupling or shifting the actual charge to an amplifier at 
the periphery of the array. This eliminates the need for the long polysilicon and 
metal busses that degrade the signal with their associated resistance and 
capacitance. However, the low noise of the CCD requires the imager to be read in 

R370273.1 

2~ 




-2 



a fixed format and once the charge is read it is destroyed. The requirement of 
coupling the collected photon charge from the pixel to the periphery amplifier 
(a.k.a. CTE), requires proprietary processing steps not compatible with industry 
standards CMOS or BiCMOS processes. 
5 Solid state imaging devices have developed in parallel with CMOS 

technology and as a result all imager manufacturers developed their own 
proprietary processes to maximize imager performance characteristics and wafer 
yield. Specialized silicon wafer processing kept imager prices relatively high. 
Linear active pixel sensors have been commercially produced since 1985. 

10 Beginning in the early 90' s the move to transfer the proprietary processes to an 
industry standard CMOS processes was on. The advantages of using an industry 
standard process include: competitive wafer processing pricing, and the ability to 
provide on chip timing, control and processing electronics. By the end of the year 
1992 a 512 x 512 CMOS compatible, CID imager with a preamplifier and CDS 

15 per column had been fabricated. The imager could either be operated as a random 
access 512x512 CID, or all the columns could be summed together and operated 
as a linear active pixel sensor. 

Area arrays utilizing active pixel sensors in which a photodiode or 
photogate is coupled to an output source follower amplifier which in turn drives a 

20 Correlated Double Sampling (CDS) circuit, where the two outputs of the CDS cell 
then drives two more source followers circuits that in turn are fed into a 
differential amplifier are shown in Patent No. 5,471 ,5 15. This uses source 
follower circuits, that typically have gains less than unity that vary from one 
source follower to another. The source follower gain variation is due to variations 

25 of FET thresholds. The source follower gain variation results in a pixel to pixel 
gain mismatch. Also, the active pixel sensors suffer gain variations due to the 
CDS circuit per column, when the CDS employs a source follower pair to drive its 
output. The resulting CDS signal and its corresponding offset can have different 
gains that are not correctable by the differential amplifier. Also, the source 

30 follower configuration of active pixel doesn't allow for binning of pixels. 

The voltage mode of operation of prior art does not allow for binning, 
which, is the summation to two or more pixel signals at once. 
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What is needed is an imager device which has the low noise level of a 
CCD, the random access, and binning of a CID, and uniform gain and response 
from all pixels. 
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Conventional Approaches to Industrial/Scientific Cameras 



The CCD sensor and camera electronics technology has evolved over the 
last 30 years to meet most of these demands. However the resulting cameras 
require a state-of-the-art, large pixel, multi-port CCD chip plus several extra chips 
and usually several circuit boards filled with electronics to accomplish this. Thus, 

10 the cameras cannot physically fit in certain applications, the power consumed is 
significant, and last but not least, the resulting cameras are far too expensive for 
many applications. The necessary recombination of the video data from several 
ports, further increases the video processing complexity and ultimately drives-up 
the cost and size of the video system. 

1 5 Over the past several years, thanks to design rule shrinkage, image sensors 

using sub-micron CMOS process technology have become practical. By using 
CMOS technology for the sensor array itself, the problem of integrating extra 
circuitry on chip becomes straightforward. Elements such as A/D converters, 
timing generators, control circuitry and interface circuitry can easily be added. In 

20 addition, the operation of CMOS imagers is simplified by the elimination of the 
need for precise timing and level control of multiple clock required to drive the 
large capacitance transfer gates inherent in CCD's. Even with all of these factors, 
including the exceptional speed and pico-second gate delays of sub-micron 
processes, the analog video bandwidth per port hasn't changed much over the past 

25 20 years. 

Active pixel sensors (APS) have been proposed as the means to achieve 
the flexible benefits of CMOS cameras on a chip. Unfortunately, there are 
performance issues with the fundamental APS approach that limit its performance 
and functionality. While these limitations may be acceptable for consumer 
30 imaging applications, the demands of scientific and industrial applications have, 
up until now, been largely unmet by CMOS image sensors. 




Brief Description of the Drawings : 

Figure 1 is a prior art double polysilicon active pixel sensor; 

Figure 2 is an active column sensor in accordance with this invention; 

Figure 3 is an implementation of a pixel in accordance with the invention; 
5 Figure 4 is a schematic illustration of a matrix of pixels connected to 

incorporate a full operational amplifier per pixel forming an Active Column 
Sensor; 

Figure 5 is a view of a traditional CCD and CMOS sensor method of 
increasing video bandwidth with multiple ports; 
10 Figure 6 is a view of a conventional method of driving a common video 

buss on an imager; 

Figure 7 is a view of a high speed low noise video bus; 
Figure 8 is a view of a bus for pixel reordering with a video processing 
block and a bus demultiplexer; 
15 Figure 9 is a view of a bus for pixel interpolation for increased resolution; 

and 

Figure 1 0 is a photograph of the invention. 

Detailed Description of the Invention : 

20 Before discussing the Active Column Sensor (ACS) circuit of Figure 2 of 

the present invention and described in conjunction with a discussion of ACS 
below, it will be useful to discuss the structure of a typical double-polysilicon 
active pixel sensor of prior art as shown in Figure 1 . 

In Figure 1 each pixel 50 has a photosite 60 that has an output FET 53 

25 configured as a source follower. The source follower 53 is used to drive a 

subsequent signal conditioning circuitry, such as a Correlated Double Sampled 
Circuit (CDS) 55. The gain through a source follower 53 is less than unity. If the 
source follower located at the pixel site 50 has a given gain other pixels and their 
respective source followers in the same column may or may not have the same 

30 gain. The technique relies on wafer processing for all FETs in the array to have 
the same threshold. It isn't uncommon for FET thresholds, during operation, to 
vary by lOOmV for a linear active pixel array. 
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e active pixel 50 of the prior art includes a photogate 60 and a transfer 
gate 62 thatare used to couple photo generated charge onto the floating diffusion 
node 52 which is connected to the gate 56 of source follower 53. The drain of the 
output FET 53 i\canneCTed dltectly to a power supply rail VDD. The source 
follower output FE^Ts^ViJupyconnected to the source 56 of row access FET 58. 
When the row accessNKB^ 58 is selected for reading, the FET 58 is turned on, 
allowing output FET 53\£o be connected to a load 18 and drive the CDS circuitry 
55 directly. 

Figure 2 is a schematic diagram of a pixel 12 in accordance with the 
present invention in which the threshold variations from pixel to pixel of the prior 
art are eliminated. All pixels 12 in a row or column are in parallel and for 
simplicity only one is shown. Pixel 12 which can consist of any photosensitive 
device 10 is coupled to an FET 15 to isolate the pixel from the readout circuitry. 
The FET 15 is one FET of a differential input pair of an operational amplifier 30 
that includes FET 24. For simplicity, in Figure 2 the amplifier circuit 30 is 
configured as a positive feedback unity gain amplifier. A feedback path 32 
connects the output of amplifier 30 to input 17 which in this case is the gate of 
FET 24. The amplifier 30 could be configured to have gain, a full differential 
input or any operational amplifier configuration as the application required. The 
fixed gain of amplifier 30 eliminates the gain variability of the prior art. The 
output of the unity gain amplifier is connected to a Correlated Double Sampler 
(CDS) which is utilized to eliminate any fixed pattern noise in the video. 

A current source 20 comprising an FET 22 has its source connected to a 
power source VDD and its drain connected to the sources of differential input 
FETs 15 and 24. 

The drains of input FETs 1 5 and 24 are connected to a current mirror 
formed from FETs 26 and 28. The gates of FETs 26 and 28 are connected 
together and to the source 1 8 of input FET 15. The sources of FETs 26 and 28 are 
connected to a negative power source, VCC. 

Th^Chirce 30 of FET 24 is the output of the differential pair and is 
connected toJHpS 34. 

The input FET 1 5 could be either a N channel or P channel FET as the 
application requires. The pixel 80 could be either a photogate or photodiode. 
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Figure 3 is a detailed schematic of pixel 1 2 of the active column sensor 
shown in Figure 2. In this implementation a photogate 76 is utilized. Selection 
and reset of a sense node 72 is controlled by an FET 76. This Active Column 
Sensor pixel eliminates the separate selection/access FET 58 of prior art. All 
5 biasing and controls signals are supplied from the periphery of the pixel array. 

The pixel can be operated in the following manner. An N type substrate is 
used and the substrate is biased the most positive potential, e.g. 5.0 volts. The 
photogate 70 preferably a layer of polysilicon is biased to an integrate level (e.g. 
ft.O volts). The region 80 under the photogate 70 is depleted and as light strikes 

1 0 the immediate area, it will collect (integrate) photon generated carriers. Photogate 
72 is been biased to the 5.0 volts and will not collect photon generated carriers 
during the integration because it is biased to the same potential as the substrate. 
Photogate 72 is biased by selecting control FET 76 with the reset/Select Control 
signal. In this configuration control FET 76 is a P channel FET that is selected by 

15 a negative signal relative to the substrate, for example 0.0 volts. During 

integration FET 76 is selected, the photogate is biased by the reset/select bias that 
preferably is at 5.0 volts. After a predetermined integration time period the pixel 
is read. 

Reading the pixel is preferably accomplished in the following manner. 

20 The reset/select control is changed to 2.5 volts, causing the region beneath 

photogate 72 to be depleted, and the background level is read. Reset/select FET 
76 is turned off by setting the reset/select control to 5.0 volts. Photogate 70 has its 
potential removed, and in this example 5.00 volts. Reading the signal will occur 
as the collected photon generated charge transfers from the region beneath 

25 photogate 70 to the region beneath photogate 72. The transferred photon 

generated charge modulates the gate of input FET 15, according to the amount of 
collected. 

Fixed Pattern Noise (FPN) can be eliminated from the video information 
by utilizing CDS circuit 34. The first sample applied to the CDS circuit is the 
30 background level. The signal information is then applied to the CDS. The 
difference of the two signals provides for a fixed pattern noise free signal. 

Figure 4 is a schematic diagram of an array of pixels in accordance with 
this invention. A plurality of pixels 90a, 90b, 90c form a first column of the array, 
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and similar columns 92a-c and 94a-c complete the array. Within each column, the 
pixels are connected with their output FETs in parallel, the combination forming 
the first one of the differential input pair of operational amplifier 30. In all other 
respects, amplifiers 30a, 30b and 30c are identical to Figure 2. Each amplifier 30 
5 is connected to a CDS 34a, 34b, and 34c respectively. The outputs of CDS 34a, b, 
c are connected through column select switches 96a, 96b, and 96c, the common 
terminals of which are connected to output buffer 98 which can be a source 
follower, or a more complex signal conditioner as required by the specific 
application. 

10 Industrial and scientific imaging applications require much higher 

performance and functionality than that required for consumer imaging products. 
Typical applications of this class of cameras include machine vision for automated 
inspection. Many of the applications require high readout speeds for video rate or 
even faster imaging without sacrificing image quality. In addition to image 

1 5 quality, the applications have come to demand greater functionality in the camera. 
Features such as flexible shuttering and electronic zoom, random access and 
selectable region of interest for maximizing frame rates and minimizing data 
storage (especially useful in tracking applications). Lowering the cost of machine 
system development is the recent advancement of single chip CMOS cameras. 

20 Newly developed CMOS cameras have all the flexibility previously listed; 
however, the analog video bandwidth per port hasn't changed from the traditional 
CCD, CID or Photodiode technologies. This invention relates to high speed video 
bus requirements in general and more specifically to solid state imagers. 

The following section describes one embodiment of an on-board, high- 

25 speed bus that allows pixel rates exceeding those of single port CCD or APS 
devices in accordance with the present invention. This bus in combination with 
the highly parallel nature of amplifier per column techniques of CMOS sensors 
that provides both the functionality and high speed performance required for 
scientific and industrial applications. Experimental results of one embodiment of 

30 the present invention are also presented. 
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1. Conventional Video Bus issues 
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Most mega-pixel image sensors, including both CCD imagers and APS 
imagers, have a maximum pixel rate inadequate to meet the frame rate needs of 
industrial and scientific imaging. CCD devices are limited by both clocking rates 
and the speeds of the Correlated Double Sampled (CDS) circuitry. In addition the 
higher amplifier bandwidth required for higher pixel rates increases noise levels. 
With the column parallel nature of CMOS imagers, the amplifier and CDS can be 
run at the line rate rather than the pixel rate. The video bandwidth constraints 
come in terms of the multiplexing speed. CMOS imagers typically multiplex their 
signals onto a common analog video bus. The more signals that are multiplexed or 
switched onto the bus, the greater the capacitive load of that bus. Therefore, as 
more signals are connected to the bus, the bandwidth of the bus is reduced. 
Alternatively, greater power is needed to charge and discharge the bus with its 
associated capacitance to maintain bandwidth. In order to overcome the 
constraints, designers of CCD's and APS sensors have resorted to dividing. up the 
imager into halves, quarters, or smaller groupings of sub-imagers, jammed 
together. The signal from each of these sub-imagers is brought out to its own port, 
as shown in Figure 5. Circuit 2 of Figure 5 is a video port that is replicated four 
times in this example. This approach has been used to provide high frame rate 
devices, or even to meet standard frame rates with large mega-pixel imagers. This 
adds system size, complexity, power and cost to handle the multiple analog 
amplifier chains. In addition, it is an extremely challenging task to balance the 
amplifier chains completely over all possible pixel rates and temperatures. This 
issue has become even more of a problem in recent years as imagers have grown 
larger, now up to full wafer size. The process variations across an array can lead 
to further balance problems, and even variations in noise characteristics, due to 
process variations across a wafer. 

Often CMOS imagers have column parallel amplifiers that drive a 
common video bus. The common video bus is seen mostly as a capacitive load to 
each individual column amplifier as shown in Figure 6, circuit 14. In order for 
each amplifier to truly represent the pixel value onto the common video bus, the 
amplifier must charge or discharge the bus with in one pixel time constant. It must 
be stable long enough for a sample and hold circuit (or similar) to accurately 
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present the resultant signal to an A to D converter. A conservative engineer will 
want at least 5t (tau or time constants) to accurately allow the video bus to settle 
the video value presented by each individual column amplifier. At higher video 
bus speeds the individual column amplifier, Figure 6, circuit 10, is unable to 
5 properly charge or discharge the video bus and results in a loss of contrast ratio. 
At higher pixel element rates where the contrast ratio is compromised, the 
individual column amplifier characteristic and the video switch characteristics 
begin to affect the resultant video. The individual column amplifiers will have 
slightly different offsets, Figure 6, circuit 10, with slightly different drive 
10 capabilities and each video switch, Figure 6, circuit 12, will have slightly different 
^ resistances and slightly different thresholds. This combination of column 

3 amplifier and video switch characteristics results in each column amplifier having 

1 different time constants relative to charging and discharging the video bus. The 

'.1 column amplifier and video switch are common to every pixel in that column. 

'"4 15 Thus, variations in the video switch characteristics result in what appears to be 

column based Fixed Pattern Noise (FPN). As more columns are added, each 
=: video switch adds more associated capacitance, Figure 6, circuit 14, due to the 

U source and drain junctions of MOSFET or Bipolar transistors. The more columns 

added to the bus, the higher the total capacitance. Having identified the source of 
^ 20 column based FPN and knowing that the pixel element rates cannot exceed the 

ability of the weakest column amplifier on the video bus a solution should be 
identifiable, without requiring high power amplifiers per column. 



2. High Speed PVS Bus 

25 A preferred approach over the segmented four port imager of Figure 5, 

would be to read out the imager four times faster on a single port. The ACS series 
of imagers incorporates a novel method of increasing the read rate of analog buses 
called the PVS bus. This speed increase has been achieved without increasing 
power consumption, and full video bandwidth is maintained even at the higher 

30 speeds. Common mode noise pickup is eliminated through the use of fully 
differential processing. 
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The high speed, low noise, low power analog PVS bus for imagers as 
shown in Figure 7 utilizes either standard sequential or random access decoders 
for selection of a particular column. In addition to selecting that column, it has the 
added preselection circuitry to pre-select the next three columns in the read 
5 sequence in parallel. The number of columns pre-selected can be scaled to meet 
the application requirements; four buses are used in Figure 7 for illustration 
purposes only. Preselection, circuitry 20 of Figure 7 is effectively a four input "or" 
Boolean logic function. Now instead of having separate quadrants of pixels as in 
Fig 5, there is only now only one. Preselection circuitry works in conjunction 
10 with parallel multiplexors, where the four columns are selected in parallel and are 
multiplexed to a separate differential video bus, Figure 7, circuit 26, for each 
column selected. Parallel multiplexors, Circuit 22, Figure 7. as shown in Figure 7, 
circuit 26 utilizes differential video per column and has two video buses for each 
column address. Therefore the total number of video buses is eight for Figure 7. 

Vhe purpose of selecting the current column and pre-selecting the next 
three columns is to allow the video bus to charge up to the proper value and settle 
prior to bemg demultplexed by as shown in Figure 7 circuit 24. By pre-selecting 
the three coijum^Yor pixel time constants) ahead of time, the column video 
processing i^Quitrv /on ly has to drive the video bus at one quarter the actual pixel 
20 read rate /one-fo^fti the bandWidth) and therefore can be made smaller and lower 
power man th^N^puld oraerwise have to be. Also, since each column is 
connected to only on^ouj/of every four columns the video bus has only one-fourth 
of the capacitanceHs^cafcise there are only one fourth of the transmission gates (or 
switches) to drive. As\a result, each column selected also pre-selects the next 
25 three columns in sequenceX The column selection sequence remains conventional, 
with out the need for post processing reconstruction of the original image required 
of multi-port imagers. The Vialog pre-charging is done at a one-fourth the 
bandwidth in Figure 7 than the ptior art conventional single ended video bus and 
only the demultiplexing is done at tite normal bandwidth. 
30 The demultiplexer of Figure 7, 24 is a fully differential video bus 

designed to match the fully differential multiplexor of FIGURE7, circuit 26. 
Parallel multiplexors isolate all the different video buses. The timing for the 
demultiplexer circuit is generated via external control, an on chip counter, a small 





- 11 - 



shift register, or a controller. The demultiplexer timing control logic which in the 
implementation shown, is a counter with count decode logic. The video 
processing block that takes the differential video, provides gain, offset, filtering, 
or any video processing function. The demultiplexer timing control logic may 
5 decode the video bus circuit in any sequence that the application requires. 

By utilizing the highly parallel nature of CMOS imagers and more 
specifically the fully differential video provided by the Active Column 
Sensor(ACS patent approved) technology, the PVS Bus is able to maintain near 
ideal MTF with high video bandwidths. The parallel nature of the PVS Bus also 
10 allows pixel reordering as shown in Figure 8, circuit 30. Pixel reordering can be 
!=? column and/or row based. An example of row and column based reordering is the 

3 Bayer (or other) color filter. Figure 8 illustrates how the PVS Bus demultiplexer 

3 can go to another demultiplexer to reorder pixels as columns are read. The 

7! manner that the pixels are reordered can be row dependent, as is the case for color 

M 15 filters. 

- J 3. Combining Columns 

U Another useful approach to the column parallel nature of CMOS imagers 

Z is the ability to select multiple columns at once to average the video signal. This 

3 20 is possible because the column amplifiers of Figure 6, circuit 10 are identical in 

every detail and when more than one is selected at once the outputs from each 
amplifier try to drive each other and the result is that the two or more signals are 
averaged. This allows for higher speed of operation and it also gives a new 
method of binning or interpolating pixels. Binning is a term used to combine two 
25 or more pixel signals together. The higher speed of operation is due to two or 
more amplifiers driving the same video bus and as a result there is up to two times 
(or more) the ability to drive the same amount of capacitance. The binning is a 
result of combining two or more signals at the same time on the same video bus. 
The net result of combining multiple signals(a.k.a. averaging) onto the same bus is 
30 to effectively a multi-resolution imager. 

This multi-resolution ability of the video bus can be used to effectively 
increase the resolution, through interpolation, not just decrease the resolution 
through binning. The increase of resolution is through the use of selectively 

/z 
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binning adjacent signals in between reading individual signals. In this manner, the 
time sequence would be for a conventional column orientated CMOS video bus, 
as in Figure 6, would be select column 1 and read the signal. Next in sequence 
leave column 1 selected and also select column 2. Read the combined (binned or 
5 averaged) signal and deselect column 1 and now only read column 2 signal. In 
this manner for every two adjacent signals read, a third interpolated signal can be 
read effectively increasing the resolution through interpolation. This allows a 
typical imager with 640 columns to have the effective resolution of 1279 pixels 
through interpolation. 

10 

4. Row Binning 

The ACS series of imagers has been developed specifically for scientific 
and industrial applications and utilizes a bus in accordance with the present 
invention. The design incorporates the Active Column Sensor design approach 

15 that uses a unity gain amplifier per column with a selectable input transistor at 
every pixel along that column. An ACS technology imager will uniquely allow 
binning or interpolation along the row in a similar manner as the columns were 
just described above. The ability to bin (or average) is described in the ACS 
imager patent. Interpolation that the ACS technology uniquely allows, would 

20 utilize the just described method in the previous paragraph for the first row of 
video. The ACS imager would then select two adjacent rows of video (rows 1 and 
2) and the above interpolation would be repeated and followed again by only 
selecting row 2 and rereading only row 2. In this manner a typical imager with 
480 rows, can have an interpolated resolution of 959 rows. 

25 What has just been described in the previous two paragraphs can be done 

at a much higher rate with the bus in accordance with the present invention 
described herein. Utilizing the PVS Bus where the current pixel being read and 
the next three(or more) pixels are pre-selected and pre-charging the remaining 
video buses in parallel, the column(pixel) averaging is done by the demultiplexor. 

30 Where the demultiplexor of Figure 7, circuit 24, is used to select adjacent pixels at 
the same time. Figure 9 illustrates in circuit 52 how simple the added circuitry 
can be to allow column to column interpolation. 
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While the invention has been described in connection with a presently 
preferred embodiment thereof, many modifications and changes may be apparent 
to those skilled in the art without departing from the true spirit and scope of the 
invention, which is intended to be defined solely by the appended claims. 



